ENH: Add an errors flag to tz_localize #13058
Conversation
|
You need to add some tests to test this code path. |
Current coverage is 84.14%@@ master #13058 diff @@
==========================================
Files 137 137
Lines 50261 50261
Methods 0 0
Messages 0 0
Branches 0 0
==========================================
Hits 42288 42288
Misses 7973 7973
Partials 0 0
|
jreback
added
Missing-data
|
|
||
|
|
||
| - Add ``errors`` flag to tz_localize, so you can silently ignore nonexistent timestamps and replace them with ``NaT`` if ``errors`` is set to ``'coerce'``. The default behaviour is still raising a NonExistentTimeError (``errors='raise'``) |
There was a problem hiding this comment.
double backticks on tz_localize and NonExistenTimeError. add the issue number as well.
|
pls add some tests. |
| @@ -3959,10 +3968,12 @@ def tz_localize_to_utc(ndarray[int64_t] vals, object tz, object ambiguous=None): | |||
| int64_t v, left, right | |||
| ndarray[int64_t] result, result_a, result_b, dst_hours | |||
| pandas_datetimestruct dts | |||
| bint infer_dst = False, is_dst = False, fill = False | |||
| bint infer_dst = False, is_dst = False, fill = False, is_coerce = errors=='coerce', is_raise = errors=='raise' | |||
There was a problem hiding this comment.
put these on another line, we don't PEP these yet
|
If needed, I will add some tests for DatetimeIndex tomorrow... |
|
@jreback hey, I think I discovered a bug while writing more detailed test cases for Timestamp.tz_localize. Namely, for some US timezones, the up to 7am local time on the day DST has started, the time is shifted by 1 hour in advance >>> pd.Timestamp('2014-03-09 06:00', tz='US/Eastern')
Timestamp('2014-03-09 07:00:00-0400', tz='US/Eastern')
>>> pd.Timestamp('2015-03-08 06:01', tz='US/Eastern')
Timestamp('2015-03-08 07:01:00-0400', tz='US/Eastern')
>>> pd.Timestamp('2014-03-09 06:00', tz='US/Pacific')
Timestamp('2014-03-09 07:00:00-0700', tz='US/Pacific')Note that this doesn't happen after 7am that day, or in the following days >>> pd.Timestamp('2014-04-09 06:00', tz='US/Eastern')
Timestamp('2014-04-09 06:00:00-0400', tz='US/Eastern')
>>> pd.Timestamp('2015-04-08 06:01', tz='US/Eastern')
Timestamp('2015-04-08 06:01:00-0400', tz='US/Eastern')
>>> pd.Timestamp('2014-04-09 06:00', tz='US/Pacific')
Timestamp('2014-04-09 06:00:00-0700', tz='US/Pacific')Am I missing something, or is this really a bug? I have pushed my tests. Here is |
|
@dancsi that last is this bug: #7825 when a Well for a PR on that. (its independent of this issue). |
|
well for that issue you can add your examples, but also take the examples from that issue as well. This PR should be independent of the other issue. IOW your tests should be testing this specific issue. |
|
I don't understand why codecov is failing? After all, I added some tests, and they are passing... |
|
you have to rebase against master, I fixed the .yml file. |
| @@ -533,6 +533,21 @@ def test_ambiguous_nat(self): | |||
| di_test = DatetimeIndex(times, tz='US/Eastern') | |||
| self.assert_numpy_array_equal(di_test, localized) | |||
|
|
|||
| def test_nonexistent_raise_coerce(self): | |||
| from pytz.exceptions import NonExistentTimeError | |||
There was a problem hiding this comment.
add the issue number as a comment
|
looks pretty good. couple of commets. pls rebase / squash. ping on green. |
|
@jreback all checks have passed. Thank you for guiding me through the process :) |
|
very nice @dancsi thank you! if you would like to work on other issues would be great! |
closes #13057
Author: dancsi [email protected]